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ABSTRACT 

Criterion- referenced measurement (CRM) has received 
increasing attention in regular education. However, it is in 
education for handicapped children that CRM’s flexibility for 
individualization of both instruction and evaluation become even more 
fully realized. Research is described on one of the first CRM systems 
(Individual Achievement Monitoring System: lAMS) ever devised 
exclusively for the handicapped and designed for widespread 
implementation. Methodological problems are discussed, such as 
inappropriateness of item sampling, difficulties in retention 
testing, and determination of adequate criterion levels of mastery 
for handicapped children. Flexibility of research findings based on 
CRM is also examined. (Author) 
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Cr I ter I on- referenced measurement (CRM) has re- 
ceived Increasing attention In regular education. 
However, It Is In education for handicapped children 
that CRM's flexibility for Individualization of both 
Instruction and evaluation become even more fully 
realized. Research Is described on one of the first 
CRM systems (individual Achlevament Monitoring 
System: I AMS) ever devised exclusively for the handi- 
capped and designed for widespread Implementation. 
Methodological problems are discussed, such as Inap- 
propr lateness of Item sampling, difficulties In re- 
tention testing, and determination of adequate cri- 
terion levels of mastery for handicapped children. 
Flexibility of research findings based on CRM Is 
also examined. 
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INTRODUCTION 

There is a greav. deal of interest In criterion-referenced measure- 
ment (CRM) at present. This year's convention alone reflects this ever- 
increasing attention. Several advantages to CRM seem to be fostering Its 
growth.. First, some like the Idea that CRM, unlike norm- referenced measure- 
ment CNRM) , does not force the test interpreter into making sometimes in- 
appropriate group comparisons. An individual is treated as an Individual 
In the testing interpretation. The student is referenced toward the behavior 
to be measured, rather than pre-established group norm performance. In 
effect, CRM affords individualized evaluation for individualized instruction. 
Second, the focus of CRM is on the subject content to be mastered, not on a 
vast array of numerical subscore continuums which mean little In and of 
themselves. Simple go- no go decisions are made on the basis of some mastery 
level that Is stipulated for competency In the subject area, and concepts 
learned and those not yet mastered can be described In a simple fashion to 
educators and pa<^ents alike (Ml liman, 1970 ). Third, to build adequate measur- 
ing devices, and in turn to relate them back to the Instructional process, the 
educator Is forced to plan exactly what is to be taught by statino .**ry s^iecitic 
behavioral objectives. Fourth, the fact that CRM mastery levels must be set 
forces the teacher to examine more intensively just what is and is not es- 
sential for continuous educational progress, since her go-no go decisions 
based upon CRM data affect directly the child's movement along the instruc- 
tional. continuum. Additional advantages could be listed. Because they are 
found so often In tho literature, no formal review will be attempted here 
(see, for example, Bolvln 6 Glaser, 1968 ; Cox S Sterrett, 1970 ; Gorth, Grayson, 
Popejoy S Stroud, 1969 ; Johnson 6 Kress, 1971 » Popham, 1970 ; Shoemaker, 1971 ; 
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Simon, 1969). 

This paper will attempt to define what are believed to be distinguish- 
ing features of handicapped children which demand specialized measurement 
systems, not merely quick revampings of existing normal-child measurement 
technology* ("Handicapped" In this paper Is to be Interpreted as limited or 
Impaired mental functioning.) A CRM system designed especially for the 
handicapped will be described. Finally, some CRM Issues specific to the 
handicapped are listed. It Is the contention of this paper that all too 
often the efforts of measurement specialists get directed toward the solution 
of problems for normal children, when In reality more general Izable schemes 
for all children could be produced If constraints for mentally handicapped 

children were kept In mind. 

FACTORS THAT HAVE LED TO 
ANTIQUATED MEASUREMENT PROCEDURES 
FOR THE HANDICAPPED 

All of the .four major advantages of CRM described above have applicability 
to any "type" of child, whether he be normal, mentally retarded, emotionally 
disturbed, or whatever. However, there are some additional considerations 
peculiar to handicapped children that measurement experts rarely pay heed to. 
This situation Is brought about not because measurement people are Incapable 
of adapting their tools to handicapped populations, but rather because the 
whole area of "special" or "exceptional" education for the Impaired child Is 
neglected by most educational researchers. As a result, the many areas of 
Impairment under the broad umbrella of "education for the handicapped 
suffer Innovative implementation lags much longer than even with the normal 

education realm. 
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First* there is a widespread belief that any student monitoring fof the 
handicapped can be adequately approached by wise selection of standardized 
tests to match in-house instructional objectives, proper level of content 
difficulty, etc. However, beyond the purpose of screening and classifying 
handicapped children as being deviant from normal populations, little value 
can be derived from standardized tests for populations, monitoring children in 
an on-going fashion. While broad objectives might be matched from the in- 
structional program to those of a standardized test, they are never specific 
enough to ferret out the root problems of a handicapped child. It matters 
not whether the standardized test is an achievement device or a diagnostic one. 
The standardized tests simply are too global in nature to be of much use to 
remediation of the child. Tests interpreted in a CRM sense and built to re- 
flect the very specific, local school system objectives are much more useful 
to the handicapped children specialist. In suiranary, the usual global ob- 
jectives, which seem to suffice in locating trouble spots for normal children, 
simply are not specific enough to pinpoint difficulties of mentally handi- 
capped children. 

A second point in connection with the misuse of standardized tests In 
guaging academic progress of handicapped students Is the use of norm- referenced 
or group-oriented interpretations. As stated above, NRM devices have their 
appropriate role In Identifying a "static'* type of devlancy from the normal. 
However, it Is one thing to Identify a child as being, say, mentally retarded, 
and quite another to Judge his progress ("dynamic") In terms of NRM. One 
already knows the child is deviant, and he will gain little by showing that 
his progress Is also deviant ("A rose Is a rose"!). Rather, we need for 
tne deviant child a mode of Interpretation that references his performance 
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to the criterion tasks and competency levels (CRM). 

One strong bias In special educators and regular educators alike when 
It comes to building any type of detailed (In-depth) measurement system Is 
that there just Is not much to pick up at all. So why bother with any 
measurement methodology? Most special educators simply cannot see the value 
of building formal monitoring or accountability systems for mentally handi- 
capped pupils. This sorry attitude has led to almost total lack of account- 
ability In special education programs (see Proger, 1971) • 

CONSTRAINTS UPON 
MEASUREMENT SYSTEMS 
FOR THE HANDICAPPED 

In dealing with several measurement experts from the balllv;lck of regular 
education. It seems next to Impossible for them to see what educators of the 
handicapped consider are unique testing problems. In part, this lack of com- 
munication is caused by the special educators who often find that they them- 
selves cannot define precisely what they think Is "special" about measurement 
In the world of special education (handicapped children). Special educators 
try to get across to evaluators that Individualization of Instruction Is of 
prime Importance to education for the handicapped and that -elaborate Item- 
examined sampling systems or any type of global-objective assessment simply 

is not applicable. "But," say the evaluators, "Individualization of In- 

% 

structlon Is a prime goal of all education, not just for the handicapped. 

The IPI and CAM monitoring systems have fit In beautifully with Individualized 
Instruction." And this Is as far as the argument usually gets,' with the 
special educators wondering what they missed and the evaluators thinking they 
have hit the nail right on the head. 
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Unfortunately, there Is more to the Issue than meets the eye. There 
are two main considerations here. To discuss these, consider the two most 
common class plans for the handicapped. First, there Is the small-group, 
self** contained class. Usually having no more than ten children, the self- 
contained class handles on a full-time, year-round basis those handicapped 
children who cannot keep up with their regular education counterparts. 

Second, there Is the resource room which accoimiodates less severely handi- 
capped children on a part-time basis; most of their time is spent In a 
regular class with only specialized Individualization given in the resource 
room on a one-to-one basis. 

In both classes, the two primary distinguishing features of Instruction 
that directly affect measurement systems are (a) the Individualization re- 
quires a more Intensive subject-content-diagnostic In regular education or- 
ientation than (b) the Individualization process not only allows different 
rates of progress, but also demands the flexibility whereby everybody in the 
one self-contained class or resource room might be on a totally different 
Instructional approach (this Is rarely ever the case in so-called individual- 
ized systems for normal children). 

With respect to the first distinguishing feature, special educators 
usually rely upon very precise diagnostic Information. This type of data 
simply Is not forthcoming with typical global assessment packages or sampling 
schemes. Special educators of the mentally handicapped need very specific 
diagnostic Information not even given by so-called diagnostic tests In reading. 
For example, consider the letter "a". It Is one thing to learn whether a 
retarded child can visually discriminate the "a" from configurationally similar 
distractors, or whether he can perform a similar discrimination task auditor- 
lly. One can usually get this type of data from various diagnostic reading 
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tests. However, it Is quite another thing to worry about whether the child 
can deal with *'a** embedded In words, such as In the Initial, medial, and 
final positions; whether the handicapped child might be better able to deal 
with "a" In visual modes of communication than auditory; whether the child 
has problems In copying or mimicking "a"; whether the child can deal ef- 
fectively with the difference between long and short "a" and all exceptions 
thereto; and so on. Thes e types of data cannot be obtained from the usual 
diagnostic reading tests; CRM Is one vehicle appropriate to assessing such 
basic skills. 

With regard to the second feature. In a regular classroom, even If In- 
dividualization Is practiced as well as preached. It Is doubtful whether there 
will ever no^ be a common core of Instructional sequence regardless of the 
pacing of students. Thus, for regular education, sampling schemes or Just 
assessment of very global objectives does seem appropriate. However, there 
Is just no vn, these testing approaches can accommodate a totally different 
Instructional package for every child In a single class. 

Besides worrying about the nature of Instruction with handicapped 
children the testing process Itself poses some constraints. The mentally 
handicapped child Is hindered to a much greater extent by sensory processing 
difficulties than are normals. That Is, one child might function very poorly 
In the visual modality and yet quite Intactly with regard to the auditory 
one. Thus, one definitely wants to worry about assessing at least auditory 
and visual Inputs on any task the child Is given. This alone makes the 
testing game complicated enough. While It goes without saying that Individ- 
ualized test administration must be used (group tests are virtually meaning- 
less with such groups as the retarded), this Is not enough. Because of the 
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processing problems in the central nervous system, the perceptual problems, 
motor impairments, and so on, paper-and-penci 1 responding is out of the 
question for a lot of handicapped children. Fine motor coordination of any 
type (writing, darkening blank spaces, etc.) is usually a problem. Thus, 
pointing responses, vocal replies, etc., are perhaps more appropriate. Again, 
this can be done only in an individualized testing situation. 

THE INDiVlDUAL ACHIEVEMENT 
MONITORING SYSTEM (I AMS) FOR HANDICAPPED 

CHILDREN 

A CRM system fdr the handicapped was developed jointly under the auspices 
of PRISE (Pennsylvania Resources and Information Center for Special Education) 
and NRRC/P (National Regional Resource Center of Pennsylvania), for the NRRC/P 
project itself. NRRC/P is funded under Title VJ_ " C of the Elementary and 
Secondary Education Act of 1965 * The NRRC/P project has four divisions: Urban 
(Philadelphia), Middle Urban (Harrisburg and environs), Suburban (Philadelphia 
suburbs), and Rural (area around University Park, Pa. — home of Pennsylvania 
State University). The CRM monitoring system, called the Individual Achieve- 
ment Monitoring System (lAMS), is being developed and operated in the Su- 

3 

burban Division. 

For the advantages usually cited for CRM, as well as the special benefits 
for handicapped children even moreso than for normals, the entire accountabil- 
ity system was centered around CRM. Standardized tests are still used in the 
classic program evaluation designs, but these aspects are of minor interest 
to project personnel. In deciding to opt for a CRM accountability system, 
the first step was to examine currently functioning CRM-oriented programs 
that might possibly be adapted to the handicapped children's needs of NRRC/P. 
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The two main existing systems of CRM-orlented evaluation/ Instruct I on that 
were considered feasible were Individually Prescribed Instruction (IPI) and 
Comprehensive Achievement Monitoring (CAM). 

Because IPI requires adherence to certain guidelines on the part of any 
school agency considering Implementing the system (evidently to comply with 
their own research data-gathering needs), and because NRRC/P needed a great 
deal of flexibility In trying to make necessary modifications In a CRM net- 
work for the handicapped, IPI was ruled out. Turning toward CAM, NRRC/P 
personnel could not see how an Item-examinee sampling framework could yield 
detailed enough pictures of Individual children to make highly personalized 
programming decisions. In several classes for the handicapped within the 
umbrella of the NRRC/P auspices, group Instruction Is so rarely used that 
any type of £jj[ou^ evaluations are meaningless. While many other factors 
entered Into the decision not to adapt either CAM or IPI, It should be clear 
why the decision was made to build a CRM system using curriculum and eval- 
uation expertise found within NRRC/P. Thus, "Phase One" of the lAMS dealt 
with getting a CRM system Into practice as soon as possible for the 1971“1972 
academic year. 

While NRRC/P personnel found that a total existing CRM system could not 
be adapted readily to the project, an effort was still made to economize 
time, effort, and manpower by trying to adapt not entire systems, but com- 
ponents of systems. One crucial component Is a coherent system of behavioral 
objectives. Thus, It was thought feasible to make the rounds of existing col- 
lections of behavioral objectives. Unfortunately, the classification systems 
of these objectives were simply too gross to a’ low specific Individualized 
prescriptions and the appropriate CRM testing 'o go along with It. With all 
' the talk and funded projects that have dealt with behavioral objectives, this 
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inability to tie in with a total systematic effort was quite frustrating. 

Not only were the collections of objectives not written within a complete 
classification framework, but also the gaps here and there at various levels 
made any linkage with NRRC/P unfeasible. A second component that NRRC/P 
thought it might be able to adapt In lieu of a total existing CRM system 
was a series of test items to match the very specific objectives. Several 
objectives collections had test items appended to the objectives. Again, 
the items simply did not cover specific enough behavior. Also, the auditory- 
visual processing problems mentioned earlier would not be accoiwnodated. 

The general procedure, of trying to adapt existing project components to 
form a new total instructicnal/evaluation system just was not feasible. There 
was also the occasional outbreak of "project paranoia" whereby people are 
reluctant to release materials "not quite completed," "in revision," and so 

on. 

Thus, sadly enough, Phase One of the I AMS effort required building on 
Its own a bank of objectives and test items to mesh with the type of diag- 
nostic individualization peculiar to education of the mentally handicapped. 

In NRRC/P, children whose functioning levels of reading and arithmatlc fall 
within the K to 6 range are admitted to its classes. The mentally handi- 
capped children usually show some severe deficits in reading and/or arithmetic, 
and thus all instructional and remedial efforts in NRRC/P classes focus upon 
these two content areas. To simplify discussion, only the reading section of 

the lAMS will be described. To begin the task of building a very specific, 

linguistic- 

coherent system of behavioral objectives, a^phonics program (Glim, i960) was 

selected to serve as a model for a realistic instructional sequence that any 

« 

teacher could tie Into diagnostically. For purposes of diagnostic teaching, 
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objectives were written at the most specific level possible (e.g*» "The 
child will mimic the /m/ sound presented orally," or, "The child will 
distinguish words with /m/ as the Initial sound from words with other sounds 
In Initial position presented orally.") Since a large number of handicapped 
children who have problems with reading must be helped at the basic skills 
level, these specific objectives are exactly what are needed. For each of 
these objectives so generated, multiple-choice auditorily-oriented or 
visually-oriented test Items were written. 

The CRM-guided Instructional system was geared to three types of testing: 
placement. Immediate achievement, and retention. The Immediate achievement 
tests examine every specific objective of two-week chunks of Instruction. 

That Is, the curriculum sequence was divided ahead of time by curriculum experts 
Into what they thought would be two-week periods of Instruction. Thus, If a 
child simply Is not getting much from the Instructional process, he Is dis- 
covered early enough before his learning problems multiply Irreparably. A 
test, or monitor. In the I AMS system may run In length from eighty to about 
one hundred fifty Items. Because of the Intensity of this monitoring, the 
achievement monitors serve as measures of both global attainment and, In the 
case of failure, of diagnostic assessment. For retention, four two-week 
units of Instruction were combined Into an eight-week retention module, where 
sampling of objective- 1 tern mappings was employed. The length of a retention 
monitor, after sampling has been employed, reduces to that of a single achieve- 
ment monitor. For the time being, the retention tests also serve as placement 
tests, the third category of lAMS testing (standardized diagnostic and achieve- 
ment tests are also used to aid In placement decisions). 
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For purposes of Intra* and Inter-project reporting, the very specific 
objectives described above are combined Into what Is akin to course ob- 
jectives (e.g., "The ability to Identify, name, reproduce, and use In con- 
tent the letter /s/ In all Its forms, and In various positions In a given 
word.") There are usually from three to five of these course objectives for 
a unit of Instruction. Percentage mastery scores are computed Immediately 
after each testing by the teachers themselves. Feedback Is therefore put to 
use when It can still make a difference. A criterion mastery level of 80^ 

Is set for total monitor achievement. In general, If a child reaches cri- 
terion on the specified body of material (two weeks' worth), he proceeds 
to the next Instructional module or unit. If the child falls to reach cri- 
terion, he Is put Into an Instructional branching network of either additional 
Instruction (parallel branching) or remediation (backwards branching). The 
decislon-making process with regard to Instructional programming Is Illus- 
trated In Tables 1, 2, and 3* 

Insert Tables 1,2,3 about here 

Thus far In this paper, only Phase One of NRRC/P's I AMS testing system 
has been considered. That Is, the time-pressed procedures for gaining an 
Immediately operational CRM system consumed all of the curricular and eval 
uatlon departments' time during the 1971-1972 academic year. To accomplish 
this end, a particular curricular sequence was selected for reading and for 
arithmetic. These sequences were task-analyzed In detail for both behavioral 
objectives, and corresponding test Items were constructed for the CRM system. 
The big drawback to this CRM construction procedure Is that the sequence 
of objectives and the sequence of test Items within the two-week achievement 
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monitors Is curriculum dependent. Phase Two of the NRRC/P monitoring efforts, 
therefore, was to build a CRM system of objectives and test Items readily 
adaptable to an^:. Instructional package. Phase Two requires that test Items 
and behavioral objectives be able to be rearranged In any sequence whatso- 
ever . 

To Implement Phase Two, old "system- I n-a-shoe-box" play was used. Every 
classroom will be provided with a large file drawer of behavioral objectives, 
test Items for CRM construction, and resource Ideas (worksheets. Instructional 
materials, etc.) to Implement the specific objectives. On one sheet of paper 
In the file system, the main mode of entry will be the behavioral objective. 
There will be a manual accompanying the file system to simplify the accession 
process. Once the teacher knows what specific objectives she wishes to work 
on In the near future, she will pull the appropriate sheets and arrange them 
In the Instructional sequence she sees fit. (The primary advantage to Phase 
Two's CRM monitoring system Is that the sequences of objectives and Items can 
be rearranged to suit whatever needs the teacher has; this was not true of 
Phase One's CRM system.) Finally, If Instructional resources are available 
(apart from trade books, readers, etc.), these are mentioned on the separate 

objective sheets. 

Phase Two will be a refinement of the existing Phase One CRM system. 
Because of the nature of NRRC/P's handicapped children, a range of objectives 
covering K to 6 Is sufficient for most needs. Phase One has dealt with K to 3. 
Thus, converting to Phase Two at the end of the 197l“1372 school year will 
require not only reworking the exl-tlng Phase One materials but also extend- 
ing them upwards through the A to 6 range. The job will be greatly simpli- 
fied by being able to obtain reading objectives for K to 6 from New York State 
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Education Department's SPPED project (System for Pupil and Program Evalua* 
tion and Development, directed by Robert P. O'Reilly). It should be noted 
that of the many objectives-orlented projects, SPPED was the only one that 
began to meet the needs of NRRC/P for handicapped children. (It was only 
after Phase One had been begun that SPPED 's availability and appropriateness 
became known . 

GENERAL ISSUES FOR 
DISCUSSION ON THE 
TOPIC OF CRM FOR 
THE HANDICAPPED 

A perennial thought-provoker in CRM discussions is the mastery level, 
pass-fail, cutoff point issue. For normal children, there appear to be two 
general routes that one can travel. First, one can set an overall mastery 
level of X$, as IP I has been doing. The child either attains at least cri- 
terion, or he does not. Go-No Go. The Phase One I AMS of NRRC/P follows 
this procedure, setting 80 ^ as the mastery level for all children to attain 
on all units of instruction. A second major cutoff point procedure is that 
of still requiring all children to attain the same mastery level on a given 
unit, but to vary the mastery levels from unit to unit, depending on the dif- 
ficulty of material, importance of the material for later successful per- 
formance, etc* This second mastery level determination process seems more 
reflective of reality but is certainly much more difficult to implement, let 
alone Justify specific levels decided upon. The specific issue for the handi- 
capped that should be raised in connection with the mastery level cutoff point 
is whether either of the above procedures is appropriate for any kind of 
mentally handicapped child. Because of their limited mental potential, can 
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such an exceptional child be asked to achieve the same criterion level of 
mastery as a normal chlldj especially for more difficult topics? It would 
seem more appropriate — although of doubtful feasibility — to set mastery 
levels for each child relative to his potential. 

With the Issue of reliability, Livingston (1972) talks about reliability 
calculated not about the norm or mean but rather about the criterion level 
one sets up. Another possible approach would be to give the criterion test 
(posttest) as a pretest, on which a reasonable amount of variability should 
still exist (If mastery Instruction has been successful, all children should 
be at criterion so that variability becomes quite restricted). Using the 
estimate of reliability (Internal consistency) gotten from the pretest ad- 
ministration of the mastery test, would there be any stunning disadvantages 
to the pretest-derived reliability estimate versus the Livingston estimate? 

Earlier In this paper In the section on “Constraints ...,“ the need for 
very specific, diagnostic monitoring was mentioned. This constraint alone 
would shed some doubt on the appropriateness of assessment of objectives 
written on a global level, as well as sampling schemes. In particular. Item- 
examinee sampling seems to be Inappropriate to handicapped populations. In 
sampling, one assumes some semblance of normality. Yet, It Is well-known 
that not only are distributions of mentally handicapped children skewed, but 
they are multi-modal (see, for example. Nelson, 1970). In other words, there 
are several different, unique distributions to the area of "retardation" 
(genetic retardation, "slowness" In normals, accidentally produced brain 
damage, etc.). In view of these considerations. Is there reaPy any justi- 
fication for any content or examinee sampling pa jterns with the handicapped? . 
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What role does time play In the monitoring of handicapped children's 

performance (academic, social or physical)? The National Regional Resource 
Center of Pennsylvania believes time of Instruction Is quite relevant (total 
time needed to teach a specific objective) but seriously doubts time (rate) 
of responding is pertinent to the legitimate evaluation of handicapped 
children's progress. For mentally limited children, power rather than speedi- 
ness seems to be the main concern of test administration. A few specific 
questions might be raised about dally (or at least frequent) monitoring that 
utilizes rate of responding as a criterion (such as the "Precision Teaching" 
movement, which began In 1965 at the University of Kansas). First, while 
an Individual teacher might be helped by recording rates and Interpreting 
graphs of those rates In her own way, one must always be cautious In Judging 
the validity of such results. With emotional behavior (such as dealt with 
In behavior modification), the teacher must worry about what times during 
the day the behavior In question Is recorded; If the behavior Is frequent 
during the day, then sampling appears to be the only feasible answer. With 
academic performance, the difficulty of content (such as arithmetic prob- 
lems) could fluctuate markedly from day to day, thus distorting the graphs 
of rates of responding. Further, the teacher has to concern herself about 
the comparability of sampled time periods from day to day; all types of 
outside contaminating Influences can affect changes In rates In any graphical 
presentation. A second major problem with rate recording Involves the ap- 
propriateness of rates of responding In comparison to absolute levels of 
performance. A child's rate graph could appear to show Improvement simply 
by making the same number of errors but In less time; this situation could 
easily occur In solving sets of arithmetic problems. Also, there Is the 
philosophical Issue of whether one should consider the Improvement of a 
disabled child In terms of quickness, or absolute quality., or both. 
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Another weird paradox or dilemma in measurement of on-going academic 
progress of the handicapped is the circular reasoning used to try to get 
at sensory deficits/learning styles. Because of the crucial ity process- 
ing difficulties play in the achievement of any handicapped child, some 
measurement specialists believe any concept (in arithmetic or reading, for 
sake of argument) should be measured in at least two different ways: audi** 

torily-based or visually-based. In the psychol inguistic processing model 
of Osgood, as modified by Kirk, McCarthy, and Kirk (1968), the auditory 
or visual emphasis to a test item can be introduced in any one or more of 
three dimensions: reception (stimulus), association (thought processes), 

or expression (response). Thus, for any item, there are eight (2X2X2) 
ways to modify the chain of events involved in a response to be primarily 
auditory or visual. This, of course, is not feasible to do in terms of 
test length alone- However, there appears to be a fallacious circularity 
of measurement reasoning to any testing of the reception-association- 
expression sequence. Starting at the top of the hierarchy, expression 
(responding) requires that both reception of stimuli and association of 
them with prior knowledge has occurred. Thus, expression is intimately 
dependent upon successful functioning in the processes of reception and ^ 

association. One can never ••eally claim he has attained anything near an 
uncontaminated measure of expression. One might respond that he can get 
out of this dilemma by partial ling outv In effect, the lower-level effects 
of reception and association. Yet, to get any measure at all of the latter 
two processes, a large degree of expression (responding) is required. How 
does one exit this measurement disaster? 
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Finally, with the handicapped, there are some interesting phenomena 
regarding internal consistency of a given test. Consider a two-way design 
with type of child (normal and handicapped) and complexity of test (unidimen- 
sional and multidimensional). Assuming internal consistency is a function 
of the population upon which the data is to be generated, one could hy- 
pothesize for unidimensional tasks greater internal consistency for normals 

f 

with intact processing mechanisms than for handicapped children. Even for 
highly homogeneous activities, for the handicapped these appear to be 
“different" as though they are not related at all. However, when one con- 
siders clearly multi-dimensional tests, one would predict internal con- 
sistency to change markedly (i.e., lower) for normals but negligibly for 
the process ing-impai red or mentally handicapped child, who already sees 
uni dimensional tests as multi-dimensional in effect. 

SUMMARY 

This paper has discussed criterion-referenced measurement (CRM) from 
a perspective other than that usually employed: handicapped children in 

distinction to normal children. It has been argued that all too often 
measurement experts try to generalize methodology used with normal child- 
ren to handicapped children. Further, it has been argued that the measure- 
ment needs of programs for the handicapped are quite different than those 
for normal children. A CRM system devised especially for the handicapped 
is described: the individual Achievement Monitoring System (lAMS). Finally, 

some CRM Issues specific to the handicapped are discussed. It is the hope 
of this paper that measurement experts will take more than a cursory look 
at various types of handicapped populations with a view toward devising 
more appropriate measurement systems. 
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FOOTNOTES 



The writing of this paper was supported by PRISE under the 
Elementary and Secondary Education Act of 1965i Title ill (Grant No. - 
R-22-H, 48-70-0003-0). However, no endorsement of the content herein 
or the practices implied thereby is to be inferred on the part of PRISE 
or its funding source, the United States Office of Education. 

^The authors would appreciate any criticisms, comments, or related 
views from others working in the CRM area. Please address all corre- 
spondence to Dr. Barton B. Proger, Director of Evaluation and Dissemina- 
tion, Pennsylvania Resources and Information Center for Special Education 
443 South Gulph Road, King of Prussia, Pennsylvania 19406. 

3 

Those desiring further information on aspects of NRRC/P other than 
the I AMS, (see footnote 2), should write to Dr. David L. Hayden, Director 
National Regional Resource Center of Pennsylvania, Department of Special 
Education, Box 91 1» Harrisburg, Pennsylvania 17126. 
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Classical Standardized Tests Used at Year's Start for 
Screening and for Global Program Evaluation Pretest 
Levels and at Year's End for Global Posttest Levels 
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FIGURE 3 

INSTRUCTIONAL BRANCHING POSSIBILITIES WITH SPECIALIZED 

PERSONNEL FOR CHILDREN WITH PARTICULARLY SEVERE. 

* 

PROBLEMS DURING INDIVIDUAL ACHIEVEMENT' MONITORING 
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